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5 Field of the Invention 

[0001] The present invention relates to rendering devices and, 
more specifically, to a rendering device which can be incorporated 
in a drive assistant device . In more detail, the rendering device 
generates a display image of an area around a vehicle based on 
10 an image that is captured by an image capture device fixedly placed 
in the vehicle. 



Description of the Background Art 
[0002] The drive assistant device incorporating such a 

15 rendering device as described above has been actively researched 
and developed. A conventional-type drive assistant device is 
mounted in a vehicle, and generally includes an image capture device, 
a rudder angle sensor, a computing unit, a rendering device, and 
a display device. The image capture device is fixedly placed in 

20 a predetermined position in the vehicle, and the image capture 
device is provided for capturing an image of an area that is defined 
by the viewing angle of the image capture device. The resulting 
image is hereinafter referred to as a captured image . The rudder 
angle sensor is also fixed in a predetermined position in the vehicle, 

25 and detects to what degree the steering wheel of the vehicle is 




turned. Based on the detection result, the computing unit 
calculates an estimated path for the vehicle to take. The 
rendering device then renders the estimated path on the captured 
image, and the image generated thereby is a display image such 
5 as the one shown in FIG. 20. The display image is displayed on 
the display device. 

[0003] With such a display image on the display device, a driver 
of the vehicle can know if his/her current steering will fit the 
vehicle in a parking space without colliding into any obstacle 

10 in a close range of the driver's vehicle. If his/her steering 
is not appropriate, the estimated path is displayed out of the 
parking space in the display image. Therefore, the driver can 
appropriately adjust the rudder angle of the steering wheel. 
[0004] There is another type of conventional drive assistant 

15 device exemplarily disclosed in Japanese Patent examined 
Publication No. 2-36417 (1990-36417). The drive assistant device 
additionally carries an active sensor for measuring a distance 
between the vehicle and an obstacle that is observed near the 
estimated path. Based on the measurement result provided by the 

20 active sensor, the computing unit determines which part of the 
estimated path is to be rendered on the captured image. The part 
which is determined to be rendered on the captured image is 
hereinafter referred to as a rendering estimated path. In this 
manner, the rendering device accordingly renders on the captured 

25 image the rendering estimated path, which ends right before the 




obstacle . 

[0005] The above-described conventional drive assistant 
devices have the following two problems. First, the estimated 
path is fixedly determined in color for display. Thus, even if 
5 the color is similar in tone to a predominant color of the display 
image, the color is unchangeable. Here, the predominant color 
is mainly determined by the road, for example, regardless of whether 
the road paved or not with asphalt . If this is the case, the driver 
finds it difficult to instantaneously locate the estimated path 

10 on the display image. 

[0006] Second, the estimated path that is rendered in the 
display image is represented simply by lines, which fails to help 
the driver instantaneously perceive how far he/she can move the 
vehicle. More specifically, as shown in FIG. 21, a vehicle Vusr 

15 carrying the conventional drive assistant device is moving toward 
an obstacle Vbst. In this case, the vehicle Vusr first collides 
into a corner point Pcnr of the obstacle Vbst, not intersection 
points Pcrg of an estimated path Pp and the surface of the obstacle 
Vbst. This means that the farthest point possible for the vehicle 

20 Vusr to move is the corner point Pcnr of the obstacle Vbst. As 
such, even if the estimated path is so rendered as to end immediately 
before the object, the second problem remains yet unsolved. 



SUMMARY OF THE INVENTION 
25 [0007] Therefore, an object of the present invention is to 
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provide a rendering device which generates display image that 
shows an estimated path in an eye-catching manner for the driver 
to easily locate. 

Another object of the present invention is to provide 
5 a rendering device which generates display image generated that 
is indicative and helpful for the driver to know how far he/she 
can move the vehicle. 

[0008] The present invention has the following features to 
attain the above-described objects. 

10 [0009] A first aspect of the present invention is directed to 
a rendering device for generating a display image of an area around 
a vehicle for drive assistance. The rendering device comprises 
a reception part for receiving a current rudder angle of a steering 
wheel of the vehicle f roma rudder angle sensor fixed in the vehicle; 

15 a derivation part for deriving an estimated path for the vehicle 
to take based on the rudder angle received by the reception part; 
and an image generation part for generating the display image based 
on a captured image which is captured by an image capture device 
fixed in the vehicle, and the estimated path that is derived by 

20 the derivation part. Here, in the display image, the estimated 
path is overlaid on an intermittent basis. 

[0010] A second aspect of the present invention is directed 
to a rendering device for generating a display image of an area 
around a vehicle for drive assistance. The rendering device 
25 comprises a first reception part for receiving a distance to an 




obstacle that is located around the vehicle from a measuring sensor 
placed in the vehicle; a first derivation part for deriving a 
farthest point for the vehicle tomove based on the distance received 
by the first reception part; a second reception part for receiving 
5 a current rudder angle of a steering wheel of the vehicle from 
a rudder angle sensor fixed in the vehicle; a second derivation 
part for deriving an estimated path for the vehicle to take based 
on the rudder angle received by the second reception part; and 
an image generation part for generating the display image based 
10 on a captured image which is captured by an image capture device 
fixed in the vehicle, the farthest point derived by the first 
derivation part, and the estimated path derived by the second 
derivation part. 

[0011] These and other objects, features, aspects and 
15 advantages of the present invention will become more apparent from 
the following detailed description of the present invention when 
taken in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
20 [0012] FIG. 1 is a block diagram showing the hardware structure 

of a rendering device Urndl according to a first embodiment of 

the present invention; 

FIG, 2 is a diagram showing a display image Sout generated 

by a processor 1 of FIG. 1; 
25 FIG. 3 is a diagram showing a position where an image 




capture device 4 of FIG. 1 is placed; 

FIG . 4 is a diagram showing a captured image Sept captured 
by the image capture device 4 of FIG. 1; 

FIG. 5 is a flowchart showing the processing procedure 
5 of the processor 1 of FIG. 1; 

[0013] FIG. 6 is a diagram showing a left-side trajectory Ppl 
and a right-side trajectory Pp2 derived in step S6 in FIG. 5; 

FIG. 7 is a diagram showing overlaying position data 
Dsp generated in step S7 in FIG. 5; 
10 FIG. 8 is a diagram showing the display image Sout 

generated in step S8 in FIG. 5; 

FIG. 9 is a diagram showing. the display image Sout 
generated in step S15 in FIG. 5; 

FIG. 10 is a block diagram showing the hardware structure 
15 of a rendering device Urnd2 according to a second embodiment of 
the present invention; 

[0014] FIG. 11 is a diagram showing a display image Sout 
generated by a processor 21 of FIG, 10; 

FIG. 12 is a flowchart showing the processing procedure 
20 of the processor 21 of FIG. 10; 

FIG . 13 is a block diagram showing the hardware structure 
of a rendering device Urnd3 according to a third embodiment of 
the present invention; 

FIG. 14 is a diagram showing a display image Sout 
25 generated by a processor 41 of FIG. 13; 
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FIGS. 15A and 15B are diagrams showing placement 
positions of active sensors 441 to 444 of FIG. 13; 
[0015] FIG. 16 is a flowchart showing the processing procedure 
of the processor 41 of FIG. 13; 
5 FIG. 17 is a diagram for demonstrating the process in 

step S43 in FIG. 16; 

FIG. 18 is a diagram for demonstrating the process in 
step S44 in FIG. 16; 

FIG . 19 is a detailed diagram showing an estimated region 
10 Rpt generated in step S410 in FIG. 16; 

FIG. 20 is a diagram showing a display image displayed 
by a conventional drive assistant device; and 

FIG. 21 is a diagram for explaining problems unsolvable 
by the conventional drive assistant device. 

15 

DETAILED DESCRIPTION OF THE INVENTION 

[0016] FIG. 1 is a block diagram showing the hardware structure 
of a rendering device Urndl according to a first embodiment of 
the present invention. In FIG. 1, the rendering device Urndl 

20 includes a processor 1, a program memory 2, and a working area 
3. The program memory 2 is typified by ROM (Read Only Memory), 
and stores a program PGa for defining the processing procedure 
in the processor 1. By following the program PGa, the processor 
1 generates a display image such as the display image Sout as 

25 shown in FIG. 2. The display image Sout shows an estimated path 




Pp for a vehicle Vusr (see FIG. 3) to take in the course of time. 
The estimated path Pp is composed of a left-side trajectory Ppl 
and a right-side trajectory Pp2 which are indicated by indicators 
Sindl and Sind2, respectively. Here, the left-side trajectory 
5 Ppl is for a left-rear wheel of the vehicle Vusr, while the right-side 
trajectory Pp2 is for a right-rear wheel of the vehicle . Further, 
the indicators Sindl and Sind2 are both objects in a predetermined 
shape (e.g., circle, rectangle) that is previously stored in the 
program memory 2 . 

10 [0017] The working area 3 is typified by RAM (Random Access 
Memory) , and used when the processor 1 executes the program PGa , 
The rendering device Urndl according to the above-described 
structure is typically incorporated in a drive assistant device 
Uastl . The drive assistant device Uastl is mounted in the vehicle 

15 Vusr, and includes at least one image capture device 4, a rudder 
angle sensor 5, and a display device 6 together with the rendering 
device Urndl. 

[0018] As shown in FIG. 3, the image capture device 4 is embedded 
in the rear-end of the vehicle Vusr, and captures an image covering 

20 an area to the rear of the vehicle Vusr. The resulting image is 
a captured image Sept as shown in FIG. 4. The rudder angle sensor 
5 detects a rudder angle 6 of the steering wheel of the vehicle 
Vusr, and transmits the rudder angle 6 to the processor 1. The 
rudder angle 6 here indicates at what angle the steering wheel 

25 is turned with respect to the initial position . The steering wheel 




is considered to be in the initial position when the steering wheel 
is not turned, that is, when the vehicle l^usris in the straight-ahead 
position. The display device 6 is typically a liquid crystal 
display . 

5 [0019] Described next is the operation of the drive assistant 
device Uastl, When the driver wants assistance from the drive 
assistant device Uastl, the processor 1 starts executing the 
program PGa . 

[0020] Refer now to a flowchart in FIG. 5 for the processing 

10 procedure in the processor 1 written in the program PGa. In FIG. 
5, the processor 1 first generates an image capture instruction 
Icptf and transmits the image capture instruction Icpt to the 
image capture device 4 (step SI) . Here, as shown in FIG. 5, the 
procedure returns to step SI after step 310 is completed, and the 

15 processor 1 generates another image capture instruction Icpt . The 
program PGa is written so that a time interval between those two 
image capture instructions Icpt is substantially a tl second. Here, 
the value of tl is selected so as to allow the display device 6 
to display the display image Sout for 30 frames per second. Herein, 

20 the image capture instruction Icpt is a signal instructing the 
image capture device 4 for image capturing. The image capture 
device 4 responsively captures a captured image Sept such as shown 
in FIG. 4, and stores the captured image 5cpt in frame memory 
{not shown) reserved in the working area 3 (step S2) . 

25 [0021] The processor 1 then watches a deriving timing Tl (step 




S3) . This deriving timing Tl is previously written in the program 
PGa, and allows the processor 1 to derive the left- and right-side 
trajectories Ppl and Pp2 once every 12 second. The value of t2 
is selected to be larger than that of tl (e.g., 0.1 second) since 
5 a change on a time base in the rudder angle 6 is small. 

[0022] In the deriving timing Tl, the processor 1 generates 
a detection instruction Idtc, and transmits the detection 
instruction Idtc to the rudder angle sensor 5 (step S4) . The 
detection instruction Idtc is a signal instructing the rudder angle 
10 sensor 5 to detect the rudder angle B . The rudder angle sensor 
5 responsively detects the rudder angle 9 , and stores the rudder 
angle 9 in the working area 3 (step S5) . 

[0023] Based on the detected rudder angle 6 , the processor 
1 derives the left- and right-side trajectories Ppl and Pp2 (step 

15 S6) . More specifically, derived by the processor 1 here are 
equations respectively for the left- and right-side trajectories 
Ppl and Pp2 under Ackermann' s model. Here, in the strict sense, 
the left- and right-side trajectories Ppl and Pp2 are defined as 
being trajectories that are traced by left- and right-rear wheels 

20 of the vehicle Vusr on the condition that the driver keeps the 
steering wheel at the currently derived rudder angle 6 , The 
left-side trajectory Ppl that is calculated by such an equation 
becomes an arc in a predetermined length. In more detail, the 
arc is a segment of a circle which is traceable by the vehicle 

25 Vusr around a center of the circle. The radius of the circle is 




equal to a distance from the center of the circle to a point having 
a rotation center of the left-rear wheel projected onto the road 
surface . The equation for the right-side trajectory Pp2 is similar 
except that the arc is tracedby the right-rear wheel, on its rotation 
5 center, of the vehicle Vusr. 

[0024] Then, the processor 1 generates overlaying position data 
Dsp indicating where to overlay the two indicators Sindl and Sind2, 
and stores the data Dsp in the working area 3 (step S7) . As an 
example, the left- and right-side trajectories Ppl and Pp2 as 

10 shown in FIG. 6 are derived in step S6, the processor 1 calculates 
two points aO and bO which are closest to the vehicle Vusr (not 
shown) on those trajectories Ppl and Pp2, respectively. The 
processor 1 then calculates a point al which is a predetermined 
distance Ad away from the point aO on the left-side trajectory 

15 Ppl, and a point bl which is also the predetermined distance 
Ad away from the point bO on the right-side trajectory Pp2. The 
processor 1 repeats the same processing until i (where i is a natural 
number being 2 or larger) sets of coordinates such as {aO, bO) , 
[al, bl) , (a(i-i), jb(i-i)) are calculated. The sets of 

20 coordinates are numbered starting from the one closest to the 
vehicle Vusr, Accordingly, as shown in FIG. 7, the overlaying 
position data Dsp including those numbered sets of coordinates 
are stored in the working area 3. 

[0025] Based on the overlaying position data Dsp and the 
25 aforementioned captured image Sept, the processor 1 then generates 
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a frame of the display image Sout on the frame memory (step S8) . 
Here, as already described with reference to FIG. 2, the display 
image Sout is the one having the indicators Sindl and 5ind2 overlaid 
on the captured image Sept. In step S8, in more detail, the 
5 processor 1 first selects, from the overlaying position data Dsp 
generated in step S7, a set of coordinates which is not yet selected 
and which are the smallest in number. In this example, since a 
set has not yet been selected, the set of [aO, bO) is now selected. 
The processor 1 then overlays the indicators Sindl and Sind2 onto 

10 the points aO and bO in the captured image 5cpt on the frame memory. 
After this overlaying process, such a display image Sout as the 
one shown in FIG. 8 is generated for one frame on the frame memory. 
[0026] The processor 1 then transfers the display image Sout 

on the frame memory to the display device 6 to be displayed thereon 

15 (stepS9). In the current display image Sout on the display device 
6, the indicator Sindl is overlaid on the point aO on the left-side 
trajectory Ppl, and the indicator Sind2 is overlaid on the point 
bO on the right-side trajectory Pp2 . 

[0027] Then, the processor 1 determines whether it is now the 
20 time to end the processing of FIG. 5 {step SIO) . If the processor 
1 determines that the processing should not end, the procedure 
returns to step SI for generating another display image Sout. By 
the time steps 81 and 32 are completed, another captured image 
Sept is newly stored on the frame memory. Then, in step S3, if 
25 the processor determines that the timing Tl has not come yet, the 




processor 1 then watches a timing T2 to change the overlaying 
positions of the indicators Slndl and Sind2 (step Sll) . Here, 
the changing timing T2 is previously written in the program PGa, 
and allows the processor 1 to change the overlaying positions of 
5 the indicators Sindl and Sind2 once every t3 second. If the value 
of T3 is set too small, the indicator Sindl moves too fast from 
the point aO to al for the driver to follow with her/his eyes on 
the display device 6. With consideration therefor, the value of 
t3 is selected to be larger than that of tl {e.g., 0.05 second). 

10 [0028] If the processor 1 determines that the timing T2 has 
not come yet, the processor 1 generates a frame of the display 
image Sout on the frame memory (step S12) . This is based on the 
captured image Sept stored in step S2 and the set of coordinates 
currently selected in the overlaying position data Dsp (in this 

15 example, the set of (aO, bO) ) . As such, the resulting display 
image Sout is also the one having the indicators Sindl and Sind2 
overlaid on the points aO and hO on the captured image Sept. Then, 
the processor 1 transfers the generated display image Sout on 
the frame memory to the display device 6 to be displayed thereon 

20 (step S13) . 

[0029] Next, in step SIO, if the processor 1 determines that 
it is now not the time to end the processing of FIG. 5, the procedure 
returns to step SI . By the time when steps SI and S2 are completed, 
another captured image Sept is newly stored on the frame memory. 
25 Then, in step S3, if the processor 1 determines that the timing 




Tl has not come yet, and in step Sll, if the processor 1 determines 
that the timing T2 is now right, the procedure goes to step S14. 
Then, the processor 1 selects, from the overlaying position data 
Dsp on the working area 3, a set of coordinates which is not yet 
5 selected and which are the smallest in number (step S14) . Since 
the set which was last selected is {aO, hO) , the set {al, bl) is 
now selected. 

[0030] Next, the processor 1 generates a new frame of the display 
image Sout on the frame memory based on the captured image Sept 

10 and the set of coordinates {in this example, the set of (al, hi) ) 
currently selected in the overlaying position data Dsp (step S15) . 
As such, as shown in FIG. 9, the resulting display image Sout is 
the one having the indicators Sindl and Sir!d2 overlaid on the points 
al and bl on the captured image Sept. Then, the processor 1 

15 transfers the generated display image Sout on the frame memory 
to the display device 6 to be displayed thereon (step S16) . 
[0031] Such steps SI to S16 are repeated until the determination 
in step SIO becomes Yes to end the processing of FIG. 5. In this 
manner, the overlaying positions of the indicators Sindl and Sind2 

20 change, in increments of the predetermined distance Ad, from the 
points aO and hO to a(i-I) and jb(i-l), respectively. Thus, the 
indicators Sindl and Sind2 are displayed as if moving in the same 
direction as the vehicle Vusr is heading towards along the left- 
and right-side trajectories Ppl and Pp2 . Advantageously, as the 

25 those indicators Sindl and Sind2 are displayed on an intermittent 



basis, the left- and right-side trajectories Ppl and Pp2 are also 
displayed on an intermittent basis on the display device 6. 
Accordingly, the left- and right-side trajectories Ppl and Pp2 
become more noticeable and are emphasized to a further degree. 
5 With such indicators Sindl and Sind2, the driver can 
instantaneously locate the trajectories Ppl and Pp2 in the display 
image Soufc. 

[0032] Further, every time the rudder angle 6 is detected by 
the rudder angle sensor 5 according to the deriving timing Tl, 
10 the processor 1 derives the left- and right-trajectories Ppl and 
Pp2 based on the current rudder angle 6 . In this manner, the 
trajectories Ppl and Pp2 displayed on the display device 6 always 
become responsive to the driver's steering. 

[0033] Note that, in the first embodiment, the changing timing 
15 72 may be variable . For example, in the case where the overlaying 
positions of the indicators Sindl and Sind2 are relatively close 
to the vehicle Vusr, the program PGa may be written so that the 
changing timing T2 comes earlier. If so, the left- and right-side 
trajectories Ppl and Pp2 become easier to notice. 
20 [0034] Further, in the first embodiment, the predetermined 
distance Ad between two successive points of aj and d(J+i) is 
constant on the left-side trajectory Ppl. Here, the value j is 
a positive integer between 0 and [i-l) . The predetermined distance 
Ad may not necessarily be constant. For example, in the case 
25 where the point aj is relatively close to the vehicle Vusr, the 

15 




program PGa may be written so that the predetermined distance A 
d is set to be relatively small so as to cause the processor 1 
to select the point a(j + l) . Conversely, the program PGa may be 
written so that the predetermined distance A d is set to be 
5 relatively large so as to cause the processor 1 to select the point 
a ( j+1) . In both cases, the left- and right-side trajectories Ppl 
and Pp2 become conspicuous to a further degree . 
[0035] FIG. 10 is a block diagram showing the hardware structure 
of a rendering device Urnd2 according to a second embodiment of 

10 the present invention. In FIG. 10, the rendering device Urnd2 
includes a processor 21, a program memory 22, and a working area 
23. The program memory 22 is typified by ROM (Read Only Memory) , 
and stores a program PGb for defining the processing procedure 
in the processor 21. By following the program PGb, the processor 

15 21 generates a display image Sout such as the one shown in FIG. 
11 . The display image Sout shows an estimated path fpof the vehicle 
Vusr {see FIG. 3) to be traced by a left-rear wheel of the vehicle 
Vusr. The estimated path Pp is displayed only during a display 
time period Pdt, which will be described later. 

20 [0036] The working area 3 is typified by RAM (Random Access 
Memory), and is used when the processor 21 executes the program 
PGb. The rendering device Urnd2 according to the 
above-described structure is typically incorporated in a drive 
assistant device Uast2. Here, as to the drive assistant device 

25 Uast2, the only structural difference from the drive assistant 




device Uastl of the first embodiment is that the drive assistance 
Uast2 includes the rendering device Vrnd2 instead of the rendering 
device £7rndi . Thus, any component illustrated in FIG . 1 has the 
same reference numeral in FIG. 10, and therefore is not described 
5 again. 

[0037] Described next is the operation of the drive assistant 
device Uast2. When the driver wants assistance from the drive 
assistant device Uast2, the processor 21 starts executing the 
program PGb in the program memory 22. 

10 [0038] Refer now to a flowchart in FIG. 12 for the processing 
procedure in the processor 21 written in the program PGb, Compared 
with FIG. 5, the flowchart of FIG. 12 includes the same steps, 
and thus those steps having the same step numbers are identical 
and thus are not described again. 

15 [0039] First, by going through steps SI to S6, the processor 
21 derives an equation for the estimated path Pp. The procedure 
then goes to step S21, and the processor 21 generates the display 
image Sout based on the captured image Sept stored in step S2 and 
the estimated path Pp derived in step S6. More specifically, the 

20 processor 21 renders the derived estimated path in its entirety 
on the display image Sout, and the resulting display image Sout 
looks like the one shown in FIG. 11. 

[0040] The procedure then goes to step S9, and the processor 
21 transfers the display image Sout currently on the frame memory 
25 to the display device 6 to be displayed thereon. Then, the 




processor 21 determines whether it is now the time to end the 
processing of FIG . 12 (stepSlO), and if the processor 1 determines 
that the processing should not end now, the procedure returns to 
step SI for generating another display image Sout on the frame 
5 memory. By the time when steps SI and S2 are completed, another 
captured image Sept is newly stored on the frame memory. Then, 
in step S3, if the processor 1 determines that the timing Tl has 
not come yet, the processor 1 then determines whether it is now 
in the display time period Pdt for the estimated path Pp (step 

10 322) . Here, the display time period Pdf is previously written 
in the program PCb, and comes every t4 second in the second 
embodiment. This means that the estimated path Pp appears on 
and disappears from the display with a time lapse of t4 second. 
Note that, if the value of t4 is set too small, the appearance 

15 and disappearance of the estimated path Pp will be too swift for 
the driver to notice. With consideration therefor, the value of 
t4 is selected to be larger than that of tl (e.g., 0.1 second). 
[0041] If the processor 21 determines that it is now in the 
display time period Pdt, the procedure goes to step S21. The 

20 processor 21 then generates, on the frame memory, the display image 
Sout including the estimated path Pp (see FIG. 11) , The procedure 
then goes to step 39, and the processor 21 transfers the current 
display image Sout on the frame memory to the display device 6 
to be displayed thereon . Then, the processor 2 1 determines whether 

25 it is now the time to end the processing of FIG. 12 (step SIO), 
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and if the processor 1 determines that the processing should not 
end now, the procedure returns to step SI for generating another 
display image Sout, In step S3, if the processor 21 determines 
that the deriving timing Tl has not come yet, and in step S22, 
5 if the processor 1 determines that the present time is not in 
the display time period Pdt, the procedure goes to step S23. In 
step 23, the processor 21 transfers, to the display device 6 to 
be displayed thereon, the captured image Sept stored in step S2 
(see FIG. 4) as the display image Sout without any changes 

10 thereto (step S23) . 

[0042] Such steps Si to S23 are repeated until the determination 
in step SIO becomes Yes to end the processing of FIG. 12. In this 
manner, the estimated path Pp is displayed only during the display 
time period Pdt. The estimated path Pp appears on and disappears 

15 from the display on an intermittent basis. Accordingly, the 
estimated path Pp becomes noticeable, and the driver finds it easy 
to locate the estimated path Pp in the display image Sout. 
[0043] FIG. 13 is a block diagram showing the hardware structure 
of a rendering device Urnd3 according to a third embodiment of 

20 the present invention. In FIG. 13, the rendering device Urnd3 
includes a processor 41, a program memory 42, and a working area 
43. The program memory 42 is typified by ROM (Read Only Memory) , 
and stores a program PGc for defining the processing procedure 
in the processor 41. By following the program PGc, the processor 

25 41 generates a display image Sout such as the one shown in FIG, 




14. The display image Sout shows an estimated region Rpt on a 
road surface Frd for the vehicle Vusr (see FIG. 3) to move. 
Specifically, the estimated region Rpt is defined by the left- 
and right-side trajectories Ppl and Pp2 described above in the 
5 first embodiment, and a line segment Llmt passing through a no-go 
point Plmt. Here, the no-go point Plmt is a point indicating the 
farthest limit for the vehicle Vusr to move, and if the vehicle 
Vusr keeps moving, the vehicle might first collide into the 
obstacle Vhst. 

10 [0044] The working area 43 is typified by RAM (Random Access 
Memory), and is used when the processor 41 executes the program 
PGc. The rendering device Urnd3 according to the 
above-described structure is typically incorporated in a drive 
assistant device Uast3, Here, as to the drive assistant device 

15 Uast3, the structural difference between the drive assistant 
device Uastl and the drive assistant device Uast3 is that the drive 
assistant device Uast3 includes the rendering device Urnd3 instead 
of the rendering device Urndl, and further includes four active 
sensors 441 to 444, which is exemplified herein as a measuring 

20 sensor. These are the only structural differences, and thus any 
component illustrated in FIG. 1 has the same reference numeral 
in FIG. 13, and therefore is not described again. 
[0045] As shown in FIG. 15A, the active sensors 441 to 444 are 
embedded in the rear-end of the vehicle Vusr, preferably, in a 

25 lateral direction. The active sensors 441 to 444 thus arranged 

20 




emit ultrasonic waves or radio waves toward the area to the rear 
of the vehicle Vusr, and monitor reflected waves. Thereby, as 
shown in FIG. 15B, distances dl to d4 to an obstacle Vbst located 
closest behind the vehicle Vusr are detected by the active sensors 
5 441 to 444. 

[0046] Described next is the operation of the drive assistant 
device UastJ. When the driver wants assistance from the drive 
assistant device Uast3, the processor 41 starts executing the 
program PGc in the program memory 42. 

10 [0047] Refer now to a flowchart in FIG. 16 for the processing 
procedure in the processor 41 written in the program PGc, In FIG. 
16, the processor 41 first generates a distance measuring 
instruction Ijnsr, and transmits the distance measuring 
instruction Imsr to all of the active sensors 441 to 444 (step 

15 S41) . Here, the distance measuring instruction Imsr is a signal 
to instruct all of the active sensors 441 to 444 to detect the 
distances dl to d4 , and to transmit those distances to the processor 
41. The active sensors 441 to 444 each responsively perform such 
detection, and store the resultant distances dl to d4 to the working 

20 area 43 (step S42) . 

[0048] Next, based on the detected distances dl to d4, the 
processor 41 calculates coordinates [xl, yl) to {x4, y4) of four 
points PI to P4 on the surface of the object Vbst (step S43) . 
Referring to FIG . 17, the process in step S43 is described in detail . 

25 FIG. 17 shows the vehicle Vusr, the obstacle Vbst, and a 




two-dimensional (2D) coordinate system. In the 2D coordinate 
system, the Y-axis connects a rotation center of a left-rear wheel 
IVri and that of a right-rear wheel JVr2. With respect to the Y-axis, 
the X-axis is a perpendicular bisector parallel to a horizontal 
5 plane. As described above, the active sensors 441 to 444 are 
securely placed in the vehicle Vusr. Therefore, positions Al to 
A4 of the active sensors 441 to 44 4 from which the ultrasonic waves, 
for example, are emitted can be all defined by coordinates {xal, 
yal) to (xa4, ya4) that are known in the 2D coordinate system. 

10 Also, angles <i) 1 to <t> 4 at which the active sensors 441 to 444 emit 
the ultrasonic waves are known. In the third embodiment, the 
angles <|> 1 to <t> 4 are formed by the X-axis and the emitted waves, 
and FIG. 17 exemplarily shows only the angle <t> 1 , As such, the 
above coordinates (xl, yl) is equal to (dl • cos 4) 1+xal , dl ' sin 

15 4>l+yal) , and those coordinates {x2, y2) to {x4 to y4) are equal 
to {d2 ' cos(l> 2-^xa2, d2 ' sin <t) 2+ya2) to {d4 ' cos <l) 4+xa4 , d4 ' sin 
<!>4+ya4), respectively. 

[0049] Then, based on the calculated four points PI to P4, the 
processor 41 calculates coordinates {xlmtf ylmt) of the corner 

20 point Pcnr of the obstacle Vbst as one example of the no-go point 
Plmt (step S44) . By referring to FIG. 18, the process in step 
S44 is now described in detail. The processor 41 first performs 
a Hough transform with respect to the points PI to P4 so that curves 
CI to C4 are derived in the Hough space which is defined by the 

25 ;o-axis and 0-axis. Here, the curves CI to C4 are expressed as 



the following equations (1) to {4), respectively. 
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[0050] According to the above equations (1) and (2), the 
processor 41 calculates coordinates (pi, 0 1) of an intersection 
point Pel of the curves CI and C2 in the Hough space, and according 
to the equations (2) to (4 ) , the processor 41 calculates coordinates 
{p2, 6 2) of an intersection point Pc2 of the curves C2 to C4 
in the Hough space . From the intersection point Pel, the processor 
41 then derives an equation for a straight line PI P2, Here, the 
line PI P2 is expressed by the following equation (5) on the 2D 
coordinate system. Similarly, a line P2 P4 is expressed by the 
following equation (6). 

y= (-cos 0 1 • x+iO 1) /sine 1 ...(5) 

y= (-COS02 • x+f>2)/sin0 2 ...(6) 

From those equations (5) and (6), the processor 41 
calculates coordinates of an intersection point of the line PI 
P2 and the line P2 P3, and the resulting coordinates are determined 
as the above-mentioned coordinates [xlmt, ylmt) . 
[0051] By similarly going through steps S4 and S5 of FIG. 5, 
the processor 41 receives the current rudder angle 9 of the vehicle 
Vusr (steps S45 and S46) . 

[0052] The processor 41 then calculates, in the 2D coordinate 




system, coordinates {xcnt, ycnt) of a center point Pent (see FIG. 
19) of the circle traceable by the vehicle Vusr when rotated {step 
S47) . The processor 41 also derives equations for circles Crl 
and Cr2, which are traced respectively by the left- and right-rear 
5 wheels Wrl and Wr2 ^ on each rotation center, of the vehicle Vusr 
when rotated around the center point Pent (step S48) . Here, since 
the coordinates (xcnt, ycnt) and the equations for the circles 
Crl and Cr2 are easily calculated under the well-known Ackermann' s 
model, steps S47 and S48 are not described in detail. Further, 

10 the circles Crl and Cr2 include the left- and right-side 
trajectories Ppl and Pp2 described above in the first embodiment. 
[0053] The processor 41 then derives an equation for a straight 
line Llmt, which passes through the coordinates {xcnr, ycnr) 
calculated in step S4 4 , and the coordinates {xcnt, ycnt) calculated 

15 in step S47 (step S49) . Herein, the straight line Llmt specifies 
the farthest limit for the vehicle Vusr to move without colliding 
with the obstacle Vbst. 

[0054] The processor 41 next generates the estimated region 
Rpt, which is a region that is enclosed by the circles Crl and 
20 Cr2 calculated in step S48, the straight line Llmt calculated in 
step S49, and a line segment Lrl2 (step S410) . Here, the line 
segment Lrl2 is the one which connects the rotation centers of 
the left- and right-rear wheels IVrl and Wr2, 

[0055] By similarly going through steps SI and S2 of FIG. 5, 
25 the processor 41 receives the captured image Sept from the image 

24 




capture device 4 (steps S411, S412) . Based on the captured image 
Sept and the estimated region Rpt generated in step S410, the 
processor 41 then generates the display image Sout on the frame 
memory. More specif ically, the processor 41 deforms the estimated 
5 region Rpt to the one viewed from the image capture device 4, and 
renders the estimated region i^pf on the captured image Sept . The 
resulting display image Sout looks like the one shown in FIG. 
14. The processor 41 then transfers the display image Sout on 
the frame memory to the display device 6 to be displayed thereon 

10 (step S414). Such steps S41 to S414 are repeated until the 
determination becomes Yes in step S415 to end the processing of 
FIG. 16. As such, as the estimated region Rpt extends to the no-go 
point Flint, the driver can instantaneously know the farthest limit 
to move the vehicle Vusr. 

15 [0056] In the first to third embodiments as described above, 
the image capture device 4 is embedded in the rear-end of the vehicle 
Vusr. The present invention, however, is not restricted thereto, 
and the image capture device 4 can be embedded in the front-end 
of the vehicle Vusr. Further, the number of image capture devices 

20 4 is not limited to one, and may be more than one depending on 
the design requirements of the drive assistant devices Uastl to 
Uast3. 

[0057] Still further, in the above-described embodiments, the 
captured image Sept is the one on which the left- and right-side 
25 trajectories Ppl and Pp2, the estimated path Pp, and the estimated 




region Rpt are rendered. Here, the captured image Sept may be 
subjected to some image processing by the processors 1, 21, and 
41 before having those rendered thereon. Such image processing 
is typified by processing of generating an image of an area around 
5 the vehicle Vusr viewed from a virtual viewpoint set high up in 
the vehicle Vusr. 

[0058] Still further, in the first to third embodiments 
described above, the captured image Sept is stored in the frame 
memory in response to the image capture instruction Icpt 

10 transmitted from the processors 1, 21, and 41 to the image capture 
device 4. The present invention, however, is not restricted 
thereto, and the captured image 5cpt is voluntarily generated by 
the image capture device 4 and then stored in the frame memory. 
Similarly, the rudder angle 6 may be detected voluntarily by the 

15 rudder angle sensor 5 without responding to the detection 
instruction Idct originating from the processors 1, 21, and 41. 
[0059] Still further, in the third embodiment described above, 
four active sensors 441 to 444 are placed in the vehicle Vusr. 
The present invention, however, is not restricted thereto, and 

20 one or more active sensors may be placed in the vehicle Vusr. Here, 
if only oneactive sensor is placed in the vehicle Vusr, thedirection 
of the lens thereof needs to be dynamically changed so that the 
angle <t> of the emitted waves is set to be wider. 
[0060] Still further, in the above-described third embodiment, 

25 the active sensors 441 to 444 are provided herein as one example 

26 



of a measuring sensor for measuring the distances dl to d4 to the 
obstacle Vbst. The present invention, however, is not restricted 
thereto, and another type of measuring sensor such as a passive 
sensor may be used. Here, to structure such an exemplary passive 
sensor, two image capture devices are required to cover the area 
to the rear of the vehicle Vusr. These image capture devices each 
pick up an image of the obstacle Vbst located behind the vehicle 
Vusr, Based on a parallax of the obstacle in images , the processor 
41 then measures a distance to the obstacle Vhst with stereoscopic 
views {stereoscopic vision) . 

[0061] Still further, in the above-described embodiments , the 
programs PGa to PGc are stored in the rendering devices Urndl to 
Urnd3, respectively. The present invention, however, is not 
restricted thereto, and those programs PGa to PGcmay be distributed 
in a recording medium typified by a CD-ROM, or over a communications 
network such as the Internet. 

[0062] While the present invention has been described in detail, 
the foregoing description is in all aspects illustrative and not 
restrictive. It is to be understood that numerous other 
modifications and variations can be devised without departing from 
the scope of the invention. 




ABSTRACT OF THE DISCLOSURE 

In a rendering device, a processor derives an estimated 
path to be traced by the left- and right-rear wheels of a vehicle 
5 based on a rudder angle that is provided by a rudder angle sensor. 
The processor then determines positions for overlaying 
indicators on the derived estimated path. The processor then 
renders the indicators on the determined points in a captured 
image which is provided by an image capture device, and generates 
10 a display image. In the display image, the indicators move along 
the estimated path in the direction the vehicle is heading towards , 
In this manner, the estimated path in the display image that is 
generated by the rendering device becomes noticeable for a driver 
of the vehicle. 
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